Video surveillance system and video surveillance method

ABSTRACT

A video surveillance system includes: a detection unit that detects a predetermined event on the basis of an image captured by a first imaging apparatus; and a control unit that controls a second imaging apparatus such that the second imaging apparatus captures an image of a predetermined position after the detection of the predetermined event.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation application of U.S. patentapplication Ser. No. 18/201,989, filed May 24, 2023, which is aContinuation application of U.S. patent application Ser. No. 17/462,808,filed Aug. 31, 2021, which is a Continuation application of U.S. patentapplication Ser. No. 16/288,195, filed Feb. 28, 2019, issued as U.S.Pat. No. 11,228,715, issued Jan. 18, 2022, which is a Continuationapplication of U.S. patent application Ser. No. 15/561,572, filed Sep.26, 2017, issued as U.S. Pat. No. 11,019,268, issued May 25, 2021, whichis a National Stage of International Application No. PCT/JP2015/059737filed Mar. 27, 2015, the disclosures of the prior applications areincorporated herein in their entirety.

TECHNICAL FIELD

The present invention relates to a video surveillance technique.

BACKGROUND ART

There are surveillance systems using a plurality of surveillance(security) cameras. Patent Document 1 proposes a surveillance method inwhich multiple cameras including a rotating camera capable ofcontrolling panning, tilting, and zooming cooperate with each other todisplay an intruding object in an enlarged manner without blind spotswith a minimum number of cameras. In this method, while an intruder isnot detected in the surveillance space, two rotating cameras captureimages of their surveillance spaces in a wide area so as to mutuallycompensate for the blind spots. When an intruder is detected in thesurveillance space on the basis of the image captured by one camera, thecoordinates on the surveillance space of the intruder are calculated,and the other camera is controlled so as to perform image capturing inthe direction of the coordinates in an enlarged manner.

Patent Document 2 proposes a surveillance camera system that causes amaster camera and a slave camera to cooperate with each other so as tocapture an image of a player who is a surveillance target object. In thesystem, when the master camera captures the image of the surveillancetarget object, on the basis of the image capturing direction of themaster camera and the position information of the master camera and theslave camera, the image capturing direction of the slave camera iscalculated. Then, by directing the image capturing direction of theslave camera to the calculated direction, the master camera and theslave camera can cooperatively capture an image of the surveillancetarget object.

RELATED DOCUMENT Patent Document

-   [Patent Document 1] Japanese Unexamined Patent Publication No.    2004-343718-   [Patent Document 2] Japanese Unexamined Patent Publication No.    2005-333628

SUMMARY OF THE INVENTION Technical Problem

In each of the above proposed methods for operating cameras (imagingapparatuses) in cooperation, images of persons are captured through aplurality of cameras such that a person such as an intruder can bemonitored without blind spots. That is, in each of the proposed methods,in a case where a certain imaging apparatus detects a person such as anintruder, another imaging apparatus is directed to the person. However,in such a method, surveillance efficiency decreases in a scene wheremultiple persons come and go. That is, in a case of applying each of theabove proposed methods to the scene where multiple persons come and go,cameras are controlled every time in response to all of many personscoming and going. Thus, the surveillance efficiency decreases.

The present invention has been made in consideration of such situations,and provides a technique for efficiently performing surveillance througha plurality of imaging apparatuses.

Solution to Problem

In aspects of the present invention, in order to solve theabove-mentioned problems, the following configurations are respectivelyadopted.

A first aspect relates to a video surveillance system. The videosurveillance system according to the first aspect includes: a detectionunit that detects a predetermined event on the basis of an imagecaptured by a first imaging apparatus; and a control unit that controlsa second imaging apparatus such that the second imaging apparatuscaptures an image of a predetermined position after the detection of thepredetermined event.

A second aspect relates to a video surveillance method executed by atleast one computer. The video surveillance method according to thesecond aspect includes: detecting a predetermined event on the basis ofan image captured by a first imaging apparatus; and controlling a secondimaging apparatus such that the second imaging apparatus captures animage of a predetermined position after the detection of thepredetermined event.

Note that, according to another aspect of the present invention, aprogram for causing at least one computer to execute the method of thesecond aspect may be employed, or a storage medium, in which such aprogram is recorded and which can be read by the computer, may beemployed. The storage medium includes a non-transitory type medium.

Advantageous Effects of Invention

According to the above aspects, it is possible to provide a techniquefor efficiently performing surveillance through a plurality of imagingapparatuses.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned objects and other objects, features and advantageswill become more apparent with reference to the preferred exemplaryembodiments to be described later and the accompanying drawings.

FIG. 1 is a diagram conceptually illustrating a hardware configurationexample of a video surveillance system according to a first exemplaryembodiment.

FIG. 2 is a diagram conceptually illustrating a processing configurationexample of a surveillance control apparatus according to the firstexemplary embodiment.

FIG. 3 is a diagram illustrating a relationship between an imagingregion of a surveillance camera and a set surveillance region.

FIG. 4 is a diagram illustrating an example of correspondencerelationship information stored in a correspondence storage unit.

FIG. 5 is a flowchart illustrating an operation example of thesurveillance control apparatus according to the first exemplaryembodiment.

FIG. 6 is a diagram conceptually illustrating an example of control ofthe surveillance camera.

FIG. 7 is a diagram illustrating a relationship between the imagingregion, a surveillance target position, and a detection target positionof the surveillance camera.

FIG. 8 is a diagram illustrating an example of correspondencerelationship information further stored in the correspondence storageunit.

FIG. 9 is a flowchart illustrating an operation example of asurveillance control apparatus according to a second exemplaryembodiment.

FIG. 10 is a diagram conceptually illustrating an example of control ofthe surveillance camera.

FIG. 11 is a diagram conceptually illustrating a processingconfiguration example of a video surveillance system according to athird exemplary embodiment.

FIG. 12 is a diagram conceptually illustrating a hardware configurationexample of the video surveillance system according to the thirdexemplary embodiment.

FIG. 13 is a flowchart illustrating an operation example of the videosurveillance system according to the third exemplary embodiment.

FIG. 14 is a conceptual diagram of a stadium to which a videosurveillance system is applied.

FIG. 15 is a diagram illustrating a specific example of a predeterminedevent.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

First, in order to facilitate understanding of exemplary embodiments ofthe present invention, the background of the present invention will bedescribed. There is a surveillance system that controls, in a case wherea certain surveillance camera detects an object such as a person or avehicle, other surveillance cameras. According to such a surveillancesystem, a plurality of surveillance cameras can comprehensively monitoran intruder or the like in cooperation with each other. However, sincesuch a surveillance system is assumed to be applied to a prohibited areain which persons are not allowed to intrude and a scene where theappearance of persons is limited to some extent, if a person is simplydetected, control of other cameras is started. Therefore, in such ascene where a large number of persons come and go or crowds are formed,such as street of a city or the like, a large-scale commercial facility,a major facility such as an airport, a terminal station, and a railwayplatform, a leisure facility, a sports facility, or a stadium, thesurveillance cameras are controlled every time in response to detectionof a person and therefore, the surveillance operation cannot beperformed efficiently.

According to exemplary embodiments of the present invention to bedescribed below, the above-mentioned problems are solved, and thus it ispossible to efficiently perform the surveillance operation. However, theexemplary embodiments to be described later are respectively justexamples, and the present invention is not limited to the configurationsof the following exemplary embodiments.

First Exemplary Embodiment

[System Configuration]

FIG. 1 is a diagram conceptually illustrating a hardware configurationexample of a video surveillance system 1 (hereinafter may be abbreviatedas a “system 1”) in the first exemplary embodiment. The system 1 has asurveillance control apparatus 10, a plurality of surveillance cameras9(#1), 9(#2) to 9(#n), and the like.

The plurality of surveillance cameras 9(#1), 9(#2) to 9(#n) include atleast one movable camera capable of changing the image capturingdirection. As long as the image capturing direction of the movablecamera can be changed, the movable direction of the movable camera mayonly be in the vertical direction or may only be in the horizontaldirection. In the following description, it is assumed that thesurveillance camera 9(#1) is a pan-tilt-zoom (PTZ) camera. The othersurveillance cameras 9(#2) and the like are fixed or movable cameras.Hereinafter, unless it is necessary to distinguish individualsurveillance cameras, each surveillance camera is referred to as a“surveillance camera 9”.

Each surveillance camera 9 is installed at a different place such thatimaging regions of itself and at least one other surveillance camera 9overlap. Each surveillance camera 9 sends a video signal (image frames)to a communication unit 5. The transmission rate of the image frames,which are sent by each surveillance camera 9 to the communication unit5, is not limited. If the transmission rate of the image frames is high,the surveillance control apparatus 10 can acquire many image frames in aunit of time. Thus, it is possible to perform highly accuratesurveillance control. The transmission rate of the image frames may bedetermined in accordance with the specification of the frame rate ofeach surveillance camera 9, the communication capacity between thesurveillance control apparatus 10 and each surveillance camera 9, theaccuracy required for the video surveillance system 1, and the like.Further, as long as each surveillance camera 9 can output a videosignal, performance and functions thereof are not limited.

The surveillance control apparatus 10 is a so-called computer, and has,for example, a central processing unit (CPU) 2, a memory 3, aninput/output interface (I/F) 4, the communication unit 5, and the likewhich are connected through a bus. Each number of hardware elements isnot limited, and these hardware elements may be collectively referred toas an information processing circuit. The hardware configuration of thesurveillance control apparatus 10 is not limited to the example shown inFIG. 1 .

The CPU 2 may also include an application specific integrated circuit(ASIC), a digital signal processor (DSP), a graphics processing unit(GPU), and the like. The memory 3 includes a random access memory (RAM),a read only memory (ROM), and an auxiliary storage apparatus (such as ahard disk).

The input/output I/F4 can connect with user interface apparatuses suchas a display apparatus 7, an input apparatus 8, a printer (not shown inthe drawing), and a projection apparatus (not shown in the drawing). Thedisplay apparatus 7 is an apparatus such as a liquid crystal display(LCD) or a cathode ray tube (CRT) display, that displays a screencorresponding to drawing data processed by the CPU 2, a graphicsprocessing unit (GPU) (not shown), or the like. The display apparatus 7may display each of images obtained from video signals sent from thesurveillance cameras 9. The input apparatus 8 is an apparatus such as akeyboard or a mouse that accepts an input of a user operation. Further,the surveillance control apparatus 10 may be mounted as a computer of amobile device (a smart phone, a tablet, or the like), and a touch panelintegrated with the display apparatus 7 and the input apparatus 8 may beconnected to the input/output I/F4.

The communication unit 5 exchanges signals with other computers anddevices through wired or wireless communication. In the presentexemplary embodiment, the communication unit 5 communicates with theplurality of surveillance cameras 9. The communication method betweenthe communication unit 5 and each surveillance camera 9 is not limited.For example, the communication unit 5 acquires the video signals fromthe respective surveillance cameras 9, and sends instruction signals tothe surveillance cameras 9. Further, a portable storage medium or thelike can also be connected to the communication unit 5.

[Processing Configuration]

FIG. 2 is a diagram conceptually illustrating a processing configurationexample of the surveillance control apparatus 10 according to the firstexemplary embodiment. As shown in FIG. 2 , the surveillance controlapparatus 10 has an acquisition unit 11, an image storage unit 12, adetection unit 13, a camera control unit 14, a correspondence storageunit 15, an output processing unit 16, an input unit 17, a calculationunit 18, and the like. These processing modules are implemented, forexample, by causing the CPU 2 to execute a program stored in the memory3. Further, the program may be installed from a portable storage mediumsuch as a compact disc (CD) or a memory card or from another computer onthe network through the input/output I/F4 or the communication unit 5,and may be stored in the memory 3.

The acquisition unit 11 acquires pieces of image data captured by therespective surveillance cameras 9, from the respective surveillancecameras 9. Specifically, the acquisition unit 11 sequentially acquiresthe pieces of image data from the video signals which are sent from therespective surveillance cameras 9. At this time, the acquisition unit 11may acquire the image data by capturing the input video signal at anoptional timing. The acquisition unit 11 stores the acquired pieces ofimage data in the image storage unit 12 in association with the piecesof identification information of the surveillance cameras 9 that capturethe images. The acquisition unit 11 acquires the images through thecommunication unit 5, for example.

The acquisition unit 11 further acquires the camera parameters throughwhich the image capturing direction of the surveillance camera 9 can bespecified, stores the camera parameters in the image storage unit 12further in association with the pieces of image data and the pieces ofidentification information of the surveillance cameras 9. The cameraparameters indicate, for example, the position of the surveillancecamera 9, the posture of the surveillance camera 9 such as the imagecapturing direction, a zoom ratio, and the like (which may includeinternal parameters), may be acquired from each surveillance camera 9,or may be acquired from the camera control unit 14. Hereinafter, in somecases, the image data may be referred to as an image.

The input unit 17 receives an input of setting of a surveillanceposition, at which a predetermined event is detected, from the user. The“surveillance position” is defined as an optional static point, line,plane, or space in the common coordinate system or the camera coordinatesystem. The “surveillance position” is, for example, “the position ofthe video surveillance line” or “the position of the surveillanceregion”. The common coordinate system is for describing coordinatesrecognized in common among the plurality of surveillance cameras 9 andamong images captured by the surveillance cameras 9. As the commoncoordinate system, for example, a global positioning system (GPS)coordinate system, a coordinate system defined on a floor map in thefacility, or the like can be used. The camera coordinate system is fordescribing coordinates which are set on an image captured by a certainsurveillance camera 9.

That is, the input unit 17 may receive the setting of the surveillanceposition through the camera coordinate system of the predeterminedsurveillance camera 9 or may receive the setting through the commoncoordinate system. In a case where the surveillance position is set inthe camera coordinate system, the input unit 17 converts thesurveillance position to the common coordinate system on the basis ofthe above-mentioned camera parameters. Various existing methods can beapplied to the conversion between the common coordinate system and thecamera coordinate system using camera parameters.

The input unit 17 outputs the received information regarding the commoncoordinate system at the surveillance position to the calculation unit18 together with the identification information of the surveillanceposition. The input unit 17 receives the setting of the surveillanceposition, which is input through the user's operation performed on theinput apparatus 8 shown in FIG. 1 , through the input/output I/F4.

Here, the video surveillance line and the surveillance region in the“position of the surveillance line” or the “position of the surveillanceregion” as a specific example of the surveillance position will bedescribed. The “video surveillance line” is a line which is superimposedon a surveillance image or the like designated by the user, and is aline which is for detecting an object crossing (passing through) theline or coming into contact with the line. The “surveillance region” isa partial region of a surveillance image or the like set by the user,and is a region for detecting a predetermined event. The surveillanceregion is also known as an area of interest (AOI), a region of interest(ROI), a sensing region, a restricted area, and the like.

FIG. 3 is a diagram illustrating a relationship between the imagingregion of the surveillance camera and the set surveillance region. Inthe example of FIG. 3 , the surveillance cameras 9(#1) and 9(#2) areinstalled such that the imaging regions thereof overlap. Further, thesurveillance region AM is set to partially overlap with an imagingregion in which two imaging regions overlap. In the example of FIG. 3 ,the surveillance region AM is set as a plane region on the commoncoordinate system, that is, a plane region on the floor (ground) in thereal world. In the present exemplary embodiment, the surveillancecameras 9 are installed, and the surveillance region AM is set, as shownin FIG. 3 .

The calculation unit 18 calculates camera parameters for capturing animage of the surveillance position for each surveillance camera 9capable of capturing the image of the surveillance position, from thesurveillance position which is output from the input unit 17 and isrepresented by the common coordinate system. The method, by which thecalculation unit 18 calculates camera parameters, is not limited, andvarious existing methods may be used. For example, the correspondencestorage unit 15 stores, in advance, a correspondence relationshipbetween the image capturable range of each surveillance camera 9 in thecommon coordinate system and the camera parameter group for capturingthe range, in addition to the stored information to be described later,for each surveillance camera 9. In that case, the calculation unit 18can determine the camera parameters such that an image of a certainposition (surveillance position) on the common coordinate system can becaptured using the correspondence relationship stored in thecorrespondence storage unit 15. The calculation unit 18 may calculatethe camera parameters such that an image of the video surveillance lineis captured at the center of the angle of view. Further, for example,the calculation unit 18 may calculate the camera parameters such thatthe ratio of the surveillance region occupying the entire field of viewis about a half of the entire region. The calculation unit 18 outputsinformation to the correspondence storage unit 15. In the information,the following are associated: the identification information (ID) ofeach surveillance position; the coordinate information of eachsurveillance position in the common coordinate system; and cameraparameters through which an image of the surveillance position of eachsurveillance camera 9 capable of capturing an image of the surveillanceposition can be captured.

The detection unit 13 detects a predetermined event in the surveillanceposition included in the image captured by each surveillance camera 9.The detection unit 13 can specify the surveillance position in the imageon the basis of the camera parameters of the surveillance camera 9 atthe time of capturing the image and the common coordinates of thesurveillance position stored in the correspondence storage unit 15. Asthe camera parameters of the surveillance camera 9 at the time ofcapturing the image, camera parameters, which are stored in the imagestorage unit 12 in association with the image, may be used. For example,the detection unit 13 can convert the camera coordinate system on theimage into the common coordinate system, on the basis of thecorrespondence relationship between the image capturable range of eachsurveillance camera 9 in the common coordinate system and the cameraparameter group for capturing an image of the range, and the cameraparameters at the time of capturing the image. For example, as describedabove, the correspondence relationship is held in the correspondencestorage unit in advance. Thereby, the detection unit 13 can determinewhether or not the surveillance position is included in the image, fromthe relationship between the range of the common coordinate system inthe image and the common coordinates of the surveillance position. In acase where the surveillance position is included in the image, thedetection unit 13 can convert the common coordinates of the surveillanceposition into the camera coordinates of the surveillance camera 9. Thedetection unit 13 detects a predetermined event at the surveillanceposition (surveillance position in the image) represented by the cameracoordinate system. However, the method of specifying the surveillanceposition of the detection unit 13 is not limited.

The predetermined events detected by the detection unit 13 may bevarious situations to be monitored in the system 1. For example, thedetection unit 13 may detect a predetermined situation which is causedby the object as the predetermined event. A “target object” is a part orthe whole of a predetermined object which is a person, an animal, avehicle such as a car, a suitcase, or the like. For example, thepredetermined event includes a target object's passing across the videosurveillance line, predetermined situations of the target object in thesurveillance region (intruding, exiting, appearing, disappearing,fighting, staying, roving, falling down, standing up, crouching down,changing the moving direction, reverse traveling, shoplifting,detouring, damaging, carrying away, leaving behind, painting graffiti,and the like), movement of the target object of a specific route definedby a line segment, and the like.

Further, the detection unit 13 may detect a predetermined situationoccurring in a predetermined number of target objects or more, as apredetermined event. The “target object” is as described above. Thepredetermined situation is a situation that is likely to be caused bythe target object, and includes, for example, the above exemplifiedsituations such as passing across the video surveillance line andintruding in the surveillance region. The predetermined event detectedin this case may be expressed as an “abnormal state” which is asituation that should be particularly noticed. For example, in a casewhere the target object is a person, when multiple persons are stayingin the surveillance region, a crowd is formed. The detection unit 13detects the state in which the crowd is formed, as an abnormal state. Inparticular, the detection unit 13 may detect, as the “abnormal state”, acase where the above-mentioned predetermined situation is caused atsubstantially the same time by a predetermined number of target objectsor more. For example, in a case where the target object is a person,when multiple persons crouch down together in the surveillance region,it is considered that an unusual situation such as a situation in whicha person is shooting a pistol has suddenly occurred. The detection unit13 detects a situation in which a large number of persons have croucheddown all at once as an abnormal state. Note that the video surveillanceline may have direction attributes (right to left, left to right, bothdirections, and the like), and may be designed such that only a casewhere there is a target object's passing in the direction specified by auser is detected as a predetermined event. Further, the videosurveillance line may be designed such that a case where the number ofpersons passing across the video surveillance line per unit time isgreater than the predetermined number is detected as a predeterminedevent. Furthermore, the video surveillance line may be designed suchthat the above state is detected as an abnormal state.

In order to detect a predetermined event, the detection unit 13 detectsa target object from the acquired image. The detection unit 13 detectsthe target object using various existing methods. For example, thedetection unit 13 detects a target object from the acquired image by abackground differencing technique. In that case, the detection unit 13constructs a model representing background information from a pluralityof images which are input in chronological order, and detects a movingobject by using the model. Most simply, the detection unit 13 defines abackground image, which is generated by averaging information of a stillregion of an image among a plurality of frames, as a background model.The detection unit 13 calculates a difference between the target imageand the background image, and detects a region having a large differenceas the target object. The detection unit 13 may directly performdetection by using a model of the target object such as a person withoutusing the background model. For example, in the case of a person, themodel used here may be a model representing the whole person or a modelrepresenting a part of a person. For example, the detection unit 13 maydetect the face or the head by using a face detector or a head detectorwhich is designed to model and detect the face or the head as a part ofa person. Alternatively, the detection unit 13 may detect the targetobject by using a detector which detects a part of a region of a personsuch as the upper half of the body or the lower half of the body.

The detection unit 13 detects predetermined events using variousexisting methods. For example, while tracking the detected target objectin a plurality of images, the detection unit 13 may detect a change ofthe target object to a predetermined state. The predetermined stateafter the change may be held in advance as an image feature value. Awell-known method may be used as the method of tracking the targetobject in the images and the method of detecting the state change of thetarget object. Alternatively, the detection unit 13 may detect apredetermined state by using the image feature value of the targetobject in the predetermined state, thereby detecting an eventcorresponding to the predetermined state.

The detection unit 13 detects a predetermined event in the surveillanceposition in the image by converting the common coordinates of thesurveillance position, which are stored in the correspondence storageunit 15, into the camera coordinate system. Therefore, when detectingthe predetermined event, the detection unit 13 can specify an ID of thesurveillance position where the predetermined event or an abnormal stateis detected, on the basis of the information stored in thecorrespondence storage unit 15. For example, the camera control unit 14to be described later uses the ID of the surveillance position where thepredetermined event or the abnormal state is detected.

After the detection unit 13 detects the predetermined event, the cameracontrol unit 14 selects the surveillance camera 9 capable of capturingan image of the surveillance position (for example, the position of thevideo surveillance line or the position of the surveillance region)among the plurality of surveillance cameras 9, and controls the selectedsurveillance camera 9 such that it captures an image of the surveillanceposition where the predetermined event is detected. The camera controlunit 14 excludes the unselected surveillance camera 9 from the controltargets at the time of detecting the predetermined event. In the presentexemplary embodiment, among the plurality of surveillance cameras 9, thecamera control unit 14 selects a surveillance camera 9, which is asurveillance camera 9 other than the surveillance camera 9 that hascaptured an image in which the predetermined event is detected, and iscapable of capturing the image of the surveillance position where thepredetermined event is detected, as a control target. For example, thecamera control unit 14 selects the surveillance camera 9 as the controltarget with reference to the correspondence relationship informationwhich is stored in the correspondence storage unit 15 exemplified inFIG. 4 .

FIG. 4 is a diagram illustrating an example of the correspondencerelationship information stored in the correspondence storage unit 15.As exemplified in FIG. 4 , the correspondence storage unit 15 storesinformation indicating a plurality of correspondence relationships ofidentification information (camera ID) of the surveillance camera 9,identification information of the surveillance position (ID of thesurveillance position), common coordinates of the surveillance position,and the camera parameters. The camera parameters, which are included inthe correspondence relationship information, indicate parameters forcapturing the image of the surveillance position which is specified bythe ID of the surveillance position of the surveillance camera 9specified by the camera ID, and indicate the posture, the zoom value,and the like of the surveillance camera 9. The common coordinates of thesurveillance position may be defined by a line segment such as the videosurveillance line and a region such as the surveillance region. In acase of the line segment, the common coordinates indicate coordinates ofboth ends of the line segment. In a case of the region, for example, thecommon coordinates indicate coordinates of the lower left vertex and theupper right vertex of the region defined as a rectangle. FIG. 4 shows anexample in which the common coordinates are set on a map of a certainstreet and are defined by X axis and Y axis.

The camera control unit 14 acquires the identification information (ID)of the surveillance position where the predetermined event is detected,from the detection unit 13. Alternatively, the camera control unit 14may specify the ID of the surveillance position where the predeterminedevent is detected, directly with reference to the correspondencerelationship information stored in the correspondence storage unit 15.That is, the camera control unit 14 can specify the ID of thesurveillance position by using the camera parameters acquired togetherwith the image captured by the surveillance camera 9(#1). For example,the camera control unit 14 selects parameters, which are closer to thecamera parameters acquired together with the image where thepredetermined event is detected, from the parameter “01” and the“parameter 02” associated with the ID “001” of the surveillance camera9(#1). The camera control unit 14 can specify the surveillance position,which is specified by the ID “002” associated with the selected“parameter 02”, as the surveillance position where the predeterminedevent is detected.

When the camera control unit 14 specifies the ID of the surveillanceposition where the predetermined event is detected, the camera controlunit 14 may acquire, from the correspondence storage unit 15, the cameraparameters and the ID of a surveillance camera 9 (control target)associated with the specified ID of the surveillance position, otherthan the surveillance camera 9 which has captured the image where thepredetermined event is detected. The camera control unit 14 controls thesurveillance camera 9 specified by the acquired ID, by using theacquired camera parameters. As a result of this control, thesurveillance camera 9 other than the surveillance camera 9 which hascaptured the image where the predetermined event is detected, capturesan image of the surveillance position where the predetermined event isdetected. The camera control unit 14 can control the surveillance camera9 to be controlled by sending the acquired camera parameters to thesurveillance camera 9 such that the camera parameters are set in thesurveillance camera 9. Further, the camera control unit 14 may transmita control signal to the surveillance camera 9 such that the posture orthe zoom value indicated by the acquired camera parameters is achieved.The camera control unit 14 may change the image capturing direction ofthe surveillance camera 9, other parameters (zoom value and the like) ofthe surveillance camera 9, or both. The specific control method is notlimited as long as the surveillance camera 9, which has not captured animage of the surveillance position where the predetermined event hasbeen detected, is controlled so as to capture an image.

The method of selecting a surveillance camera 9 to be controlled and themethod of controlling the surveillance camera 9 are not limited to theabove examples, and various known methods may be used. In thecorrespondence storage unit 15, not the camera parameters but otherinformation for enabling the surveillance camera 9 to capture an imageof the surveillance position may be associated with the camera ID andthe ID of the surveillance position. The correspondence relationshipinformation stored in the correspondence storage unit 15 may beinformation in which pieces of identification information of theplurality of surveillance cameras 9 capable of capturing images of thesame surveillance position are associated with camera parameters forcausing the surveillance camera 9 to capture images at the surveillanceposition.

Further, the correspondence relationship information, which is stored inthe correspondence storage unit 15, does not have to include informationused for controlling the image capturing direction of the surveillancecamera 9 to be controlled. In this case, the camera control unit 14 maycontrol the surveillance camera 9 to be controlled, and may sequentiallycheck the images, which are obtained from the surveillance camera 9after the control, until the target surveillance position is includedtherein. When confirming that the surveillance position as a target isincluded, the camera control unit 14 stops the control of thesurveillance camera 9.

The output processing unit 16 causes the display apparatus 7 to displaythe image which is stored in the image storage unit 12 and is capturedby each surveillance camera 9. Further, the output processing unit 16may cause the display apparatus 7 to display the image acquired by theacquisition unit 11. For example, the output processing unit 16 causesthe display apparatus 7 to constantly display the respective imagescaptured by the surveillance cameras 9. In a case where the detectionunit 13 detects a predetermined event, the output processing unit 16 maydisplay the image in which the predetermined event is detected, on thedisplay apparatus 7 with emphasis higher than the other images. Further,in a case where a predetermined event is detected, the output processingunit 16 can output to other output apparatuses (such as a printer, anaudio output apparatus, and a light emitting diode (LED)) other than thedisplay apparatus 7. In the present exemplary embodiment, the outputform of the image captured by each surveillance camera 9 is not limited.

Operation Example

Hereinafter, the video surveillance method according to a firstexemplary embodiment will be described with reference to FIGS. 5 and 6 .FIG. 5 is a flowchart illustrating an operation example of thesurveillance control apparatus 10 according to the first exemplaryembodiment. As shown in FIG. 5 , the video surveillance method accordingto the first exemplary embodiment is executed by at least one computer(CPU 2) such as the surveillance control apparatus 10. Since each stepis similar to the processing details of each of the above-mentionedprocessing modules belonging to the surveillance control apparatus 10,details of each step are appropriately omitted.

The surveillance control apparatus 10 receives an input of surveillanceposition setting by the user (S51). For example, the surveillancecontrol apparatus receives setting of the surveillance position throughthe camera coordinate system or the common coordinate system of thepredetermined surveillance camera 9. In a case where the surveillanceposition is set in the camera coordinate system, the surveillancecontrol apparatus 10 converts the surveillance position into the commoncoordinate system on the basis of the camera parameters of the camerawhich captures the image in which the camera coordinate system is set.

Subsequently, from the common coordinates of the surveillance positionobtained from the input received in (S51), the surveillance controlapparatus 10 calculates the camera parameters for capturing an image ofthe surveillance position, for each surveillance camera 9 that cancapture an image of the surveillance position (S52). The calculationmethod of the camera parameters is as described above.

The surveillance control apparatus 10 determines the ID of thesurveillance position with respect to the surveillance position set bythe input received in (S51), associates the ID of the surveillanceposition, the common coordinates of the surveillance position, and therespective camera parameters of surveillance cameras 9 calculated in(S52) with one another, and stores them in the correspondence storageunit 15 (S53). Thereby, in the correspondence storage unit 15, theidentification information (ID), the common coordinates, and the cameraparameters for performing image capturing through the surveillancecamera 9 which can capture an image of the surveillance position arestored in association with one another for the surveillance positioninput by a user.

The surveillance control apparatus 10 acquires data of the images, whichare captured by the respective surveillance cameras 9, from therespective surveillance cameras 9 (S54). The surveillance controlapparatus 10 associates the acquired image data with the identificationinformation of the surveillance camera 9 that captures the image, andstores the data in the image storage unit 12. At this time, thesurveillance control apparatus 10 further acquires the camera parametersof the surveillance camera 9 that captures the image, and stores thecamera parameters in the image storage unit 12 further in associationwith the image data and the identification information of thesurveillance camera 9.

The surveillance control apparatus 10 specifies the surveillanceposition (for example, the position of the video surveillance line orthe position of the surveillance region) in the image acquired in (S51)or the image extracted from the image storage unit 12 (S55). In thisspecification, the camera parameters acquired in (S54) and the commoncoordinates of the surveillance position stored in the correspondencestorage unit 15 are used. A method of specifying the surveillanceposition from the image is as described above. The processing of S55 isexecuted when the PTZ of the surveillance camera is controlled (when theimaging range is changed).

The surveillance control apparatus 10 detects a predetermined event atthe surveillance position, which is specified in (S55), (S56). Forexample, as the predetermined event, the surveillance control apparatusmay detect at least one of the target object's passing across the videosurveillance line, the predetermined situation of the target object inthe surveillance region, and the movement of the target object along thespecific route defined by the line segment. Further, the surveillancecontrol apparatus 10 may detect an abnormal state as the predeterminedevent. The details of the predetermined event and the detection methodof the predetermined event are as described above.

When a predetermined event at the surveillance position is detected, thesurveillance control apparatus selects the surveillance cameras 9 otherthan the surveillance camera 9 that has captured the image acquired in(S54), that can capture an image of the surveillance position specifiedin (S55), among the plurality of surveillance cameras 9 (S57). Theselection method of the surveillance camera 9 is also as describedabove.

The surveillance control apparatus 10 controls the surveillance camera9, which is selected in (S57), so as to capture an image of thesurveillance position, which is specified in (S55), (S58). For example,the surveillance control apparatus 10 changes the image capturingdirection of the surveillance camera 9, which is selected in (S57), suchthat the image of the surveillance position can be captured. The controlmethod of the surveillance camera 9 is also as described above.

FIG. 6 is a diagram conceptually illustrating an example of control ofthe surveillance camera 9. In the example of FIG. 6 , the surveillancecontrol apparatus 10 receives an input of setting of the position of thesurveillance region AM (S51). This input may be performed by the rangedesignation operation which is performed by a user on the image capturedby the surveillance camera 9(#2). The surveillance control apparatus 10calculates common coordinates of the surveillance region AM. Then, thesurveillance control apparatus 10 calculates camera parameters forcapturing an image of the position of the surveillance region AM, foreach of the surveillance cameras 9(#1) and (#2) that can capture imagesof the position of the surveillance region AM (S52). In a case where thecamera parameters of the surveillance camera 9(#2) are fixed, thesurveillance control apparatus 10 may hold the camera parameters inadvance. The surveillance control apparatus 10 associates the ID of theposition of the surveillance region AM, the common coordinates of thesurveillance region, the camera parameters of the surveillance camera9(#1), and the camera parameters of the surveillance camera 9(#2) withone another, and stores them in the correspondence storage unit 15(S53).

In (S54), the surveillance control apparatus 10 acquires the data of theimage captured by the surveillance camera 9(#2) and the cameraparameters at the time of image capturing. In a case where the cameraparameters of the surveillance camera 9(#2) are fixed, the surveillancecontrol apparatus 10 may hold the camera parameters in advance. Thesurveillance control apparatus 10 specifies the position of thesurveillance region AM in the image (S55), and detects that a person OB1intrudes into the position of the surveillance region AM, as apredetermined event (S56). At this time, the surveillance camera 9(#1)is performing image capturing in the image capturing direction D1, andis not performing image capturing of the position of the surveillanceregion AM. When the predetermined event is detected, the surveillancecontrol apparatus 10 selects the surveillance camera 9(#1) other thanthe surveillance camera 9(#2) as a surveillance camera capable ofcapturing an image of the position of the surveillance region AM (S57).Then, the surveillance control apparatus 10 controls the selectedsurveillance camera 9(#1) such that an image of the position of thesurveillance region AM is captured (S58). That is, the surveillancecontrol apparatus 10 controls such that the image capturing direction ofthe surveillance camera 9(#1) changes from D1 to D2. Thereby, both thesurveillance cameras 9(#1) and (#2) capture images of the position ofthe surveillance region AM, whereby it is possible to obtain the imagesof the surveillance region AM captured in different directions.

Advantages and Effects of First Exemplary Embodiment

As described above, in the first exemplary embodiment, the predeterminedevent is detected at the surveillance position (for example, theposition of the video surveillance line or the position of thesurveillance region) included in the image captured by a certainsurveillance camera 9. In accordance with this detection, a surveillancecamera 9 other than the surveillance camera 9, which has captured theimage in which the predetermined event is detected, is controlled so asto capture the image of the predetermined position. Thereby, two or moresurveillance cameras 9 capture images at the surveillance position wherethe predetermined event is detected. Therefore, according to the firstexemplary embodiment, it is possible to monitor, in various directions,details of the appearance of the surveillance position (for example, theposition of the video surveillance line or the position of thesurveillance region) and the appearance of the predetermined eventoccurring when the predetermined event occurs.

Further, in the first exemplary embodiment, the surveillance camera 9 iscontrolled in response to the detection of a predetermined event such asthe target object's passing across the video surveillance line, apredetermined situation of the target object in the surveillance region,or movement of the target object along the specific route defined by theline segment. Accordingly, compared with the method in which thesurveillance cameras are controlled each time a person is detected, thesurveillance work can be performed efficiently even in such a scenewhere a large number of persons come and go or crowds are formed. Inaddition, in the first exemplary embodiment, it is detected that apredetermined situation is caused by a predetermined number of targetobjects or more, and the surveillance camera 9 is controlled in responseto this detection. With such a configuration, the surveillance camera 9is controlled only depending on occurrence of an event such as anabnormal state to be particularly noticed. Thus, it is possible tofurther improve the efficiency of the surveillance work.

Further, in the first exemplary embodiment, like the PTZ camera, amovable surveillance camera 9 is included, and the movable surveillancecamera 9 is controlled so as to capture an image of the surveillanceposition where the predetermined event is detected. Accordingly, byusing the movable surveillance camera 9, it is possible tocomprehensively monitor a wide area with a small number of surveillancecameras, and it is possible to preferentially monitor the surveillanceposition such as the position of the video surveillance line or theposition of the surveillance region in conjunction with the detection ofthe predetermined event.

Second Exemplary Embodiment

In the first exemplary embodiment described above, when a predeterminedevent is detected at a surveillance position in a captured image of acertain surveillance camera 9, another surveillance camera 9 iscontrolled so as to capture an image of the surveillance position. Inthe second exemplary embodiment, when an event is detected in a capturedimage of a certain surveillance camera 9, another surveillance camera 9is controlled so as to capture an image of a position different from theposition where the event is detected. Hereinafter, a video surveillancesystem 1 according to a second exemplary embodiment will be describedfocusing on processing details different from those of the firstexemplary embodiment. In the following description, processing detailsthe same as those of the first exemplary embodiment will beappropriately omitted.

[Processing Configuration]

The processing configuration of the surveillance control apparatus 10according to the second exemplary embodiment is the same as that in thefirst exemplary embodiment (refer to FIG. 2 ). The processing details tobe described below are different from those in the first exemplaryembodiment.

The detection unit 13 detects a predetermined event at a predeterminedposition included in an image captured by a certain surveillance camera9. The camera control unit 14 controls a different surveillance camera 9so as to capture an image of a different position corresponding to thepredetermined position where the detection unit 13 has detected thepredetermined event. Hereinafter, the predetermined position, which isfor detecting the predetermined event from the image of the certainsurveillance camera 9, is referred to as a “detection target position”,and the position, which is monitored by one or more different controlledsurveillance cameras 9, is referred to as “the surveillance targetposition”. That is, the detection unit 13 detects the predeterminedevent at the detection target position which is included in the imagecaptured by the certain surveillance camera 9 and is different from thesurveillance target position. The “detection target position” is aposition predetermined for detecting the predetermined event, and is aposition of which at least one surveillance camera 9 can capture animage. The “surveillance target position” is a position to be monitoredwhich is set in association with one or more “detection targetpositions”, and is a position where at least one surveillance camera 9can perform image capturing. The “detection target position” and the“surveillance target position” are set as arbitrary static points,lines, planes, or spaces in the real world. The “detection targetposition” corresponds to, for example, the “position of the videosurveillance line” or the “position of the surveillance region” in thefirst exemplary embodiment. The “surveillance target position” is anarbitrary position of a line segment or a region. Individual commoncoordinates of the detection target position and the surveillance targetposition are acquired from the user input which is received through theinput unit 17. In the second exemplary embodiment, the correspondencestorage unit 15 stores correspondence relationships of the ID of thedetection target position, the common coordinates of the detectiontarget position, the ID of the surveillance target positioncorresponding to the detection target position, and camera parametersfor capturing an image of the surveillance target position for eachsurveillance camera 9 capable of capturing an image of the surveillancetarget position.

The predetermined event, which is detected by the detection unit 13, isas described above. However, since the surveillance target position ismonitored under the control of the surveillance camera 9, occurrence ofthe predetermined event at the detection target position becomes atrigger (priming) that is necessary for monitoring the surveillancetarget position. Therefore, for example, on the basis of such arelationship, the detection target position with respect to thesurveillance target position and the processing details of thepredetermined event to be detected are determined. For example, thedetection target position is set as a position where a person headingfor the surveillance target position is highly likely to pass. In thiscase, the detection unit 13 detects a state (person passing across thevideo surveillance line in a predetermined direction), in which theperson moves toward the surveillance target position, and a state, inwhich a person stays, as the predetermined event, at the detectiontarget position. As another example, the surveillance target position isset as a region along a rail track of a certain station platform, andthe detection target position is set as a part of the rail track aroundthe station. In this case, the detection unit 13 detects a state, inwhich the train moves toward the surveillance target position (station),as a predetermined event, at the detection target position. In thesubsequent control, for example, the surveillance camera 9, which isinstalled on the platform so as to be monitoring the escalator, iscontrolled to monitor the region along the rail track as thesurveillance target position. The surveillance target position, thedetection target position, and the types of the predetermined event tobe detected are not limited to the examples described above.

The detection unit 13 can specify the detection target position in theimage in the same manner as the surveillance position in the firstexemplary embodiment. The detection unit 13 detects a predeterminedevent in the detection target position in the image by converting thecommon coordinates of the detection target position, which is stored inthe correspondence storage unit 15, into the camera coordinate system.Accordingly, when detecting the predetermined event, the detection unit13 can specify the ID of the detection target for which thepredetermined event was detected, on the basis of the information storedin the correspondence storage unit 15. The ID of the detection targetposition, in which the predetermined event is detected, is used by, forexample, the camera control unit 14 to be described later.

FIG. 7 is a diagram illustrating a relationship between the imagingregion of the surveillance camera, the surveillance target position, andthe detection target position. In the example of FIG. 7 , thesurveillance target position is set as a plane region on the floor(ground) in the real world, and the detection target position is set asa line segment on the floor (ground) in the real world. The surveillancecamera 9(#1), the surveillance camera 9(#3), and the surveillance camera9(#4) can capture an image of the surveillance target position, and thesurveillance camera 9(#2) can capture an image of the detection targetposition. In this case, the detection unit 13 detects a predeterminedevent at the detection target position included in the image captured bythe surveillance camera 9(#2).

The correspondence storage unit 15 stores correspondence relationshipinformation as follows. The correspondence storage unit 15 stores aplurality of pieces of correspondence relationship information of theidentification information of the surveillance camera 9, theidentification information of the detection target position of which thesurveillance camera 9 can capture an image, the common coordinates ofthe detection target position, the identification information of thesurveillance target position corresponding to the detection targetposition, and the camera parameters for capturing an image of thesurveillance target position with the surveillance camera 9.

FIG. 8 is a diagram illustrating an example of the correspondencerelationship information stored in the correspondence storage unit 15.In the example of FIG. 8 , the correspondence relationship information,which is stored in the correspondence storage unit 15, indicates aplurality of correspondence relationships of the camera ID of thesurveillance camera 9, the ID of the detection target position, thecommon coordinates of the detection target position, the ID of thesurveillance target position corresponding to the detection targetposition, and the camera parameters for causing the surveillance camera9 to capture an image of the surveillance target position. The cameraparameters indicate parameters for causing the surveillance camera 9specified by the camera ID to capture an image of the surveillancetarget position specified by the ID of the surveillance target position,and indicate the posture, the zoom value, and the like of thesurveillance camera 9.

According to the example of FIG. 8 , it can be seen that a surveillancecamera 9 with a camera ID “001” can capture images of both a detectiontarget position “001” and a corresponding surveillance target position“001”. It can be seen that a surveillance camera 9 with a camera ID“002” can capture an image of the surveillance target position “001”.Further, it can be seen that a surveillance camera 9 with a camera ID“003” can capture the surveillance target position “001” and a detectiontarget position “002” and cannot capture an image of a surveillancetarget position “002” corresponding to the detection target position“002” (which can be recognized from a situation where a camera parameter“03” for capturing an image of the surveillance target position “001”and common coordinates of the detection target position “002” are heldbut camera parameters for capturing an image of the surveillance targetposition “002” are not held). It can be seen that a surveillance camera9 with a camera ID “004” can capture an image of a surveillance targetposition “002”. As described above, the information, which is stored inthe correspondence storage unit 15, is stored by receiving user'ssetting of the detection target position and the surveillance targetposition from the input unit. The calculation unit 18 calculates cameraparameters, through which an image of the surveillance target positioncan be captured, for each surveillance camera 9, on the basis of thecommon coordinates of the surveillance target position which is input bya user.

The camera control unit 14 refers to the pieces of correspondencerelationship information, which are stored in the correspondence storageunit 15, after the detection unit 13 detects a predetermined event,thereby specifying the surveillance camera 9 capable of capturing animage of the predetermined surveillance target position corresponding tothe predetermined detection target position where the predeterminedevent is detected.

For example, as described above, the camera control unit 14 acquires theidentification information (ID) of the detection target position wherethe predetermined event is detected from the detection unit 13. On thebasis of the correspondence relationship information stored in thecorrespondence storage unit 15, the camera control unit 14 specifies theidentification information (ID) of the surveillance target positioncorresponding to the acquired ID of the detection target position andthe camera ID of the surveillance camera 9 capable of capturing an imageof the surveillance target position. The camera control unit 14 canacquire the camera parameters, through which an image of thesurveillance target position can be captured, for each camera ID of thespecified surveillance camera 9. Here, in the example of FIG. 8 , it isassumed that a predetermined event is detected at the detection targetposition specified by the ID “001” of the detection target position ofthe surveillance camera 9 specified by the camera ID “001”. In thiscase, the camera control unit 14 acquires the ID “001” of thesurveillance target position corresponding to the detection targetposition ID “001”, and further acquires the camera IDs “001”, “002”, and“003” of the surveillance cameras corresponding to the ID “001” of thesurveillance target position. Further, the camera control unit 14acquires the camera parameters for each acquired camera ID. The cameracontrol unit 14 controls the surveillance camera 9, which is specifiedby the acquired ID, by using the acquired camera parameters. Forexample, the camera control unit 14 controls the surveillance camera 9with the camera ID “001” by using the camera parameter “01”, controlsthe surveillance camera 9 with the camera ID “002” by using the cameraparameter “02”, and controls the surveillance camera 9 of the camera ID“003” by using the camera parameter “03”. In this example, thesurveillance camera 9 with the camera ID “001” itself, which hasacquired the image where the predetermined event is detected, is alsocontrolled to face toward the surveillance target position.

Operation Example

Hereinafter, the video surveillance method according to a secondexemplary embodiment will be described with reference to FIGS. 9 and 10. FIG. 9 is a flowchart illustrating an operation example of thesurveillance control apparatus 10 according to the second exemplaryembodiment. As shown in FIG. 9 , the video surveillance method accordingto the second exemplary embodiment is executed by at least one computer(CPU 2) such as the surveillance control apparatus 10. Since each stepis similar to the processing details of each of the above-mentionedprocessing modules belonging to the surveillance control apparatus 10,details of each step are appropriately omitted.

The surveillance control apparatus 10 receives an input of setting ofthe detection target position and the surveillance target position froma user (S91). For example, the surveillance control apparatus 10receives the setting of the detection target position and thesurveillance target position in the camera coordinate system of thepredetermined surveillance camera 9 or the common coordinate system.When the detection target position and the surveillance target positionare set in the camera coordinate system, the surveillance controlapparatus 10 converts each of the detection target position and thesurveillance target position to the common coordinate system, on thebasis of the camera parameters of the camera that has captured the imagefor which the camera coordinate system is set.

Subsequently, from the common coordinates of the surveillance targetposition obtained from the input received in (S91), the surveillancecontrol apparatus 10 calculates the camera parameters for capturing animage of the surveillance target position, for each surveillance camera9 that can capture an image of the surveillance target position (S92).The calculation method of the camera parameters is as described in thefirst exemplary embodiment.

The surveillance control apparatus 10 determines the IDs of thedetection target position and the surveillance target position withrespect to the detection target position and the surveillance targetposition which are set through the input received in (S91), associatesthe ID of the detection target position and the surveillance targetposition, the common coordinates of the detection target position, andthe camera parameters of each of the surveillance cameras 9 calculatedin (S92), and stores them in the correspondence storage unit 15 (S93).Thus, the correspondence storage unit 15 stores the correspondencerelationship between the detection target position and the surveillancetarget position which is input from a user, the common coordinates ofthe detection target position, and the camera parameters for causing thesurveillance camera 9, which can capture an image of the surveillancetarget position corresponding to the detection target position, toperform image capturing.

The surveillance control apparatus 10 acquires pieces of image data,which are captured by the respective surveillance cameras 9, from therespective surveillance cameras 9 (S94). The surveillance controlapparatus 10 stores the acquired pieces of image data in the imagestorage unit 12 in association with the pieces of identificationinformation of the surveillance cameras 9 that capture the images. Atthis time, the surveillance control apparatus 10 further acquires thecamera parameters of the surveillance camera 9 that has captured theimage, and stores the camera parameters in the image storage unit 12further in association with the image data and the identificationinformation of the surveillance camera 9.

The surveillance control apparatus 10 specifies the detection targetposition which is set in (S91) in the image acquired in (S94) or in theimage extracted from the image storage unit 12 (S95). In thisspecification, the camera parameters acquired in (S94) and the commoncoordinates of the detection target position stored in thecorrespondence storage unit 15 are used. A method of specifying thedetection target position from the image is the same as the method ofspecifying the surveillance position according to the first exemplaryembodiment. The processing of S95 is executed when the PTZ of thesurveillance camera is controlled (when the imaging range is changed).

The surveillance control apparatus 10 detects a predetermined event atthe detection target position, which is specified in (S95), (S96). Forexample, as the predetermined event, the surveillance control apparatusmay detect at least one of the target object's passing across the videosurveillance line, the predetermined situation of the target object inthe surveillance region, and the movement of the target object along thespecific route defined by the line segment. Further, the surveillancecontrol apparatus 10 may detect an abnormal state as the predeterminedevent. The details of the predetermined event and the detection methodof the predetermined event are as described in the first exemplaryembodiment.

When a predetermined event at the detection target position is detected,the surveillance control apparatus selects the surveillance camera 9that can capture an image of the surveillance target positioncorresponding to the detection target position specified in (S95), amongthe plurality of surveillance cameras 9 (S97). The selection method ofthe surveillance camera 9 is also as described above.

The surveillance control apparatus 10 controls the surveillance camera9, which is selected in (S97), so as to capture an image of thesurveillance target position corresponding to the detection targetposition, which is specified in (S95), (S98). For example, thesurveillance control apparatus 10 changes the of image capturingdirection of the surveillance camera 9, which is selected in (S97), suchthat the image of the surveillance target position can be captured. Thecontrol method of the surveillance camera 9 is also as described above.

FIG. 10 is a diagram conceptually illustrating an example of control ofthe surveillance camera 9. The surveillance control apparatus 10receives an input of setting of the detection target position (videosurveillance line) and the surveillance target position (S91). Thisinput may be performed by the line designation operation and the rangedesignation operation, which are performed by a user, on the respectiveimages captured by the respective surveillance cameras 9. Thesurveillance control apparatus 10 calculates common coordinates of thedetection target position and the surveillance target position. Then,the surveillance control apparatus 10 calculates camera parameters forcapturing an image of the position of the surveillance target position,for each of the surveillance cameras 9(#1), 9(#3), and 9(#4) that cancapture images of the surveillance target position (S92). In a casewhere the camera parameters of any one of certain surveillance cameras 9are fixed, the surveillance control apparatus 10 may hold the cameraparameters in advance.

The surveillance control apparatus 10 stores the correspondencerelationship information in the correspondence storage unit 15 (S93), asdescribed below. For the surveillance camera 9(#2), the surveillancecontrol apparatus 10 stores correspondence relationship informationamong the camera ID, the ID of the detection target position (videosurveillance line), and the ID of the surveillance target positioncorresponding to the detection target position. For the surveillancecamera 9(#1), the surveillance control apparatus 10 storescorrespondence relationship information among the camera ID, the ID ofthe surveillance target position, and the camera parameters forcapturing an image of the surveillance target position. In a case of thesurveillance camera 9(#3), the surveillance control apparatus 10 storescorrespondence relationship information among the camera ID, the ID ofthe surveillance target position, and camera parameters for capturing animage of the surveillance target position. For the surveillance camera9(#4), the surveillance control apparatus 10 stores correspondencerelationship information among the camera ID, the ID of the surveillancetarget position, and camera parameters for capturing an image of thesurveillance target position.

In (S94), the surveillance control apparatus 10 acquires the data of theimage captured by the surveillance camera 9(#2) and the cameraparameters at the time of image capturing. In a case where the cameraparameters of the surveillance camera 9(#2) are fixed, the surveillancecontrol apparatus 10 may hold the camera parameters in advance. Thesurveillance control apparatus 10 specifies the detection targetposition (video surveillance line) in the image (S95), and detects aperson's passing across the video surveillance line, as a predeterminedevent (S96). At this time, as shown in FIG. 7 , it is assumed that thesurveillance cameras 9(#1), 9(#3), and 9(#4) have not captured images ofthe surveillance target position.

When detecting a predetermined event, the surveillance control apparatus10 selects the surveillance cameras 9(#1), 9(#3), and 9(#4) assurveillance cameras capable of capturing an image of the surveillancetarget position corresponding to the detection target position, on thebasis of the correspondence relationship information stored in thecorrespondence storage unit 15 (S97). Then, the surveillance controlapparatus 10 controls the selected surveillance cameras 9(#1), 9(#3),and 9(#4) so as to capture images of the surveillance target positionsby using the camera parameters included in respective pieces ofcorrespondence relationship information (S98). Thereby, as shown in FIG.10 , the surveillance cameras 9(#1), 9(#3), and 9(#4) capture images ofthe surveillance target position.

Advantages and Effects of Second Exemplary Embodiment

As described above, in the second exemplary embodiment, the detectiontarget position and the surveillance target position indicatingdifferent positions are acquired according to the user input, and thepredetermined event is detected at the detection target position in thecaptured image of a certain surveillance camera 9. Thus, anothersurveillance camera 9 is controlled such that an image of thesurveillance target position corresponding to the detection targetposition at which the predetermined event is detected is captured.

According to the second exemplary embodiment, for example, a precursorof something occurring at the surveillance target position is recognizedthrough detection of the predetermined event at the detection targetposition, and the surveillance target position is set under surveillanceat the stage of the precursor. For example, it is assumed that the videosurveillance line is set on a path leading to the escalator and settingis made such that an event of an abnormal state occurring is detected ina case where the number of persons passing per unit time is greater thana predetermined number. The surveillance control apparatus 10 accordingto the second exemplary embodiment recognizes a precursor of occurrenceof an accident through detection of such an event, gives a warning, andcontrols the plurality of surveillance cameras 9, which are installedaround the escalator, such that surveillance is performed focusing onthe entrance of the escalator. In such a manner, by controlling thesurveillance camera 9 before anything happens at the surveillance targetposition, it is possible to reliably monitor a situation occurring atthe surveillance target position without failing.

Modification Example

In the above-mentioned first exemplary embodiment, from the commoncoordinates of the surveillance position which is output from the inputunit 17, camera parameters for capturing an image of the surveillanceposition are automatically calculated, for each surveillance camera 9capable of capturing an image of the surveillance position (calculationunit 18). Then, the correspondence storage unit 15 stores theidentification information (ID), the coordinate information of thecommon coordinate system, and the camera parameters for eachsurveillance position. However, in the above-mentioned first exemplaryembodiment, the common coordinates of the surveillance position do nothave to be used.

For example, an input of setting of a single surveillance position ofwhich images can be captured by two movable surveillance cameras 9 isreceived through the input unit 17. In this case, a user performs anoperation of specifying the surveillance position for each of the imagescaptured by the two surveillance cameras 9. The input unit 17 acquiresthe camera coordinate information (coordinates in the image) of thesurveillance position specified by the user input for the image of theone surveillance camera 9 and the camera parameters at the time ofcapturing the image. Further, the input unit 17 acquires the cameracoordinate information (coordinates in the image) of the surveillanceposition specified by the user input to the image of the othersurveillance camera 9 and the camera parameter at the time of imagingthe image. The correspondence storage unit 15 stores the ID of thesurveillance position, the camera IDs of the two surveillance cameras 9,the camera parameters for each surveillance camera 9 for capturing animage of the surveillance position, and the camera coordinateinformation on the surveillance position in association with oneanother. By using the stored information, the detection unit 13 candetect a predetermined event in the surveillance position for each ofthe images of the surveillance cameras 9. In this case, the change ofthe camera parameters of each of the movable surveillance cameras 9 isallowed to be in the range included in the camera parameters stored inthe correspondence storage unit 15. By using the stored information, thecamera control unit 14 can control the respective surveillance cameras 9such that the cameras capture images of the surveillance position wherethe predetermined event is detected.

The second exemplary embodiment can be modified in a similar manner.That is, in the second exemplary embodiment, the common coordinates ofthe detection target position and the surveillance target position donot have to be used. In this case, the input unit 17 acquires the cameraparameters of each surveillance camera 9 when an image of thesurveillance target position is captured. Further, the input unit 17acquires the camera coordinate information (coordinates in the image) ofthe detection target position, for each of the surveillance cameras 9that can capture an image of the detection target position, and furtheracquires information in which the detection target position and thesurveillance target position are associated. The correspondence storageunit 15 stores the correspondence relationship (the pair of IDs) betweenthe detection target position and the surveillance target position, andthe correspondence relationship information between the camera ID of thesurveillance camera 9 that can capture an image of the detection targetposition or the surveillance target position and the camera parametersfor capturing an image of the surveillance target position.

Further, in a case where a user inputs the setting of the position ofthe surveillance region, the input unit 17 may acquire the image featurevalue of the surveillance region, instead of the coordinate informationof the surveillance region. In this case, the detection unit 13 detectsa region similar in the image feature value to the surveillance regionin the acquired image, and specifies the detected region as thesurveillance region. In that case, the correspondence storage unit 15may store the image feature value of the surveillance region input bythe user.

Third Exemplary Embodiment

Hereinafter, the video surveillance system and the video surveillancemethod according to a third exemplary embodiment will be described withreference to FIGS. 11, 12, and 13 . Further, the third exemplaryembodiment may be a program which causes at least one computer toexecute this video surveillance method, and also may be a storage mediumin which such a program is recorded and which can be read by the atleast one computer.

FIG. 11 is a diagram conceptually illustrating a processingconfiguration example of a video surveillance system 100 according to athird exemplary embodiment. As shown in FIG. 11 , the video surveillancesystem 100 includes a detection unit 101 and a control unit 102. Thevideo surveillance system 100 shown in FIG. 11 can be realized as theabove-mentioned surveillance control apparatus 10 shown in FIG. 1 . Inthis case, the video surveillance system 100 has the same hardwareconfiguration as the surveillance control apparatus 10 shown in FIG. 1 .

FIG. 12 is a diagram conceptually illustrating a hardware configurationexample of the video surveillance system 100 according to the thirdexemplary embodiment. As shown in FIG. 12 , the video surveillancesystem 100 may be realized as a surveillance camera 9(#n). In this case,the video surveillance system 100 (the surveillance camera 9(#n)) hasthe CPU 2, the memory 3, the input/output interface (I/F) 4, thecommunication unit 5, and the like, and the surveillance camera 9(#n)controls itself and the other surveillance cameras 9. In this case, thesurveillance camera 9(#n) is a so-called intelligent camera. Thehardware configuration of the video surveillance system 100 according tothe third exemplary embodiment is not limited to the examples in FIGS. 1and 12 , and the video surveillance system 100 may be realized by boththe surveillance control apparatus 10 and the surveillance camera 9(#n).

The detection unit 101 and the control unit 102 are realized, forexample, by causing the CPU 2 to execute a program stored in the memory3. Further, the program may be installed from a portable storage mediumsuch as a CD or a memory card or from another computer on the networkthrough the input/output I/F 4 or the communication unit 5, and may bestored in the memory 3. In a case where the video surveillance system100 is realized by both the surveillance control apparatus 10 and thesurveillance camera 9(#n), the detection unit 101 may be realized by thesurveillance control apparatus 10, and the control unit 102 may berealized by the surveillance camera 9(#2).

The detection unit 101 detects a predetermined event on the basis of theimage captured by a first imaging apparatus (for example, thesurveillance camera 9(#2)). The detection unit 101 corresponds to theabove-mentioned detection unit 13. The types of the predetermined eventand the method of detecting the predetermined event are as describedabove, and are not limited. Further, the first imaging apparatus may bea fixed surveillance camera 9 and may be a movable surveillance camera9.

After the detection unit 101 detects the predetermined event, thecontrol unit 102 controls the second imaging apparatus such that thesecond imaging apparatus (for example, the surveillance camera 9(#1))captures an image of the predetermined position. The “predeterminedposition” is a predetermined surveillance position, and is set to anarbitrary static point, line, plane or space in the real world. Thecontrol unit 102 corresponds to the above-mentioned camera control unit14. The control method of the second imaging apparatus is as describedabove, and is not limited.

As shown in FIG. 11 , the video surveillance system 100 according to thethird exemplary embodiment does not have to have the acquisition unit11, the image storage unit 12, the correspondence storage unit 15, andthe output processing unit 16 shown in FIG. 2 . These processingmodules, which are not provided in the video surveillance system 100,are provided by a different computer, and the video surveillance system100 can cooperate with these processing modules by communicating withthe different computer.

FIG. 13 is a flowchart illustrating an operation example of the videosurveillance system 100 according to the third exemplary embodiment. Asshown in FIG. 13 , the video surveillance method according to the thirdexemplary embodiment is executed by at least one computer such as thevideo surveillance system 100. For example, each step shown in thedrawing is executed by each processing module of the video surveillancesystem 100.

The video surveillance method according to the present exemplaryembodiment includes detecting a predetermined event on the basis of theimage captured by the first imaging apparatus (for example, thesurveillance camera 9(#2)) (S131), and after detection of apredetermined event (S132; YES), controlling the second imagingapparatus (for example, the surveillance camera 9(#1)) such that thesecond imaging apparatus image the predetermined position (S133).

According to the third exemplary embodiment, it is possible to obtainthe same advantageous effects as those of the first exemplary embodimentand the second exemplary embodiment described above.

EXAMPLE

Hereinafter, application examples of the video surveillance systems 1and 100 (hereinafter collectively indicated by the reference numeral 1)according to the above-mentioned exemplary embodiments will bedescribed. However, the above-mentioned application of each exemplaryembodiment is not limited to the following example.

FIG. 14 is a conceptual diagram of a stadium to which the videosurveillance system 1 is applied. For example, the above-mentioned videosurveillance system 1 is applied to a stadium where many persons gather,as shown in FIG. 14 . In this case, plural surveillance cameras 9 areinstalled at positions where images of spectators' seats, passages,entrances, and the like can be captured, and plural regions where alarge number of persons are likely to be present are respectively set inthe surveillance regions. The surveillance control apparatus 10(detection unit 13) detects that a state of a plurality of persons (acrowd) has changed all at once in the surveillance region, as apredetermined event. In the present exemplary embodiment, thepredetermined event is handled as an abnormal state. For example, as apredetermined event (abnormal state), the surveillance control apparatus10 detects that crowds, who appear in the images of the surveillancecameras 9 that capture images of the audience seats, start to run alltogether outward from a certain point.

FIG. 15 is a diagram illustrating a specific example of thepredetermined event. The example of FIG. 15 shows that a person D1 iswielding a weapon. In this case, the surveillance control apparatus 10detects that plural persons appearing in the image start to run outwardfrom the person D1 as a center, as a predetermined event.

The surveillance control apparatus 10 (the camera control unit 14) isinstalled in the vicinity of the surveillance region where thepredetermined event (abnormal state) is detected, and controls all thesurveillance cameras 9 capable of capturing images of the surveillanceregion at once such that the cameras capture images of the surveillanceregion. Thus, it is possible to capture images of the cause of theabnormal state, for example, images of a person who suddenly wields theweapon (refer to FIG. 15 ) from various directions. As a result, it ispossible to easily specify the perpetrator even in ex-post verification.

As another example, plural regions in which a large number of personsare likely to be present are respectively set in the detection targetregions, and plural surveillance target regions corresponding to thedetection target regions are set. For example, the vicinities ofspectator seats are set as the detection target regions, and pluralentrances are set as the surveillance target regions. The surveillancecontrol apparatus 10 (detection unit 13) detects that a state of aplurality of persons (a crowd) has changed all at once in the detectiontarget region, as a predetermined event (abnormal state). As in theexample of FIG. 15 , as a predetermined event (abnormal state), thesurveillance control apparatus 10 detects that crowds, who appear in theimages of the surveillance cameras 9 that capture images of thespectator seats, start to run all together outward from a certain point.

The surveillance control apparatus 10 (camera control unit 14) selects aplurality of surveillance cameras 9 capable of capturing images of thesurveillance target region (entrance) corresponding to the detectiontarget region in which the predetermined event (abnormal state) isdetected. The surveillance control apparatus 10 controls all theplurality of selected surveillance cameras 9 at once such that thecameras capture images of the surveillance target region (entrance).Thereby, even when the suspicious person escapes from the point (thedetection target region) where the suspicious person wielded the weapon,all the plurality of entrances is already set under surveillance at thattime. Thus, it is possible to reliably check appearance of thesuspicious person.

In the plurality of flowcharts used in the above description, pluralsteps (processes) are sequentially described, but the order of the stepsexecuted in each exemplary embodiment is not limited to the order ofdescription. In each exemplary embodiment, it is possible to change theorder of steps shown in the drawing within a range that does not cause aproblem in terms of the processing details thereof. Further, theabove-mentioned exemplary embodiments may be combined as long as theprocessing details do not contradict each other.

Some or all of the above processing details may be specified as follows.However, the above processing details are not limited to the followingdescription.

1. A video surveillance system including:

-   -   a detection unit that detects a predetermined event on the basis        of an image captured by a first imaging apparatus; and    -   a control unit that controls a second imaging apparatus such        that the second imaging apparatus captures an image of a        predetermined position after the detection of the predetermined        event.

2. The video surveillance system according to 1,

-   -   in which, after the detection of the predetermined event, the        control unit selects a movable imaging apparatus capable of        capturing the image of the predetermined position, from among a        plurality of movable imaging apparatuses each of which is        capable of changing an image capturing direction, controls the        selected movable imaging apparatus, and excludes an unselected        movable imaging apparatus from a control target at the time of        detecting the predetermined event.

3. The video surveillance system according to 1 or 2,

-   -   in which the detection unit detects the predetermined event at        the predetermined position which is included in the image        captured by the first imaging apparatus.

4. The video surveillance system according to 1 or 2,

-   -   in which the detection unit detects the predetermined event at        another predetermined position which is included in the image        captured by the first imaging apparatus and is different from        the predetermined position.

5. The video surveillance system according to 4, further including

-   -   a correspondence storage unit that stores a plurality of pieces        of correspondence relationship information of a relationship        between a predetermined detection target position at which the        predetermined event has been detected, and a predetermined        surveillance target position which corresponds to the        predetermined detection target position among a plurality of        predetermined surveillance target positions,    -   in which the control unit specifies the second imaging apparatus        which is capable of capturing an image of the predetermined        surveillance target position corresponding to the predetermined        detection target position at which the predetermined event has        been detected, with reference to the pieces of correspondence        relationship information, after the detection of the        predetermined event.

6. The video surveillance system according to any one of 1 to 5,

-   -   in which the detection unit detects a target object's passing        across a video surveillance line or a predetermined situation of        the target object in a surveillance region as the predetermined        event.

7. The video surveillance system according to 6,

-   -   in which the detection unit detects, as the predetermined event,        that the target object's passing across the video surveillance        line or the predetermined situation of the target object in the        surveillance region occurs for a plurality of target objects        during a predetermined time interval.

8. A video surveillance method executed by at least one computer, thevideo surveillance method including:

-   -   detecting a predetermined event on the basis of an image        captured by a first imaging apparatus; and    -   controlling a second imaging apparatus such that the second        imaging apparatus captures an image of a predetermined position        after the detection of the predetermined event.

9. The video surveillance method according to 8, further including:

-   -   selecting a movable imaging apparatus capable of capturing the        image of the predetermined position, from among a plurality of        movable imaging apparatuses each of which is capable of changing        an image capturing direction, after the detection of the        predetermined event; and    -   excluding an unselected movable imaging apparatus from a control        target at the time of detecting the predetermined event,    -   in which the step of controlling the second imaging apparatus        includes controlling the selected movable imaging apparatus as        the second imaging apparatus.

10. The video surveillance method according to 8 or 9,

-   -   in which the step of detecting of the predetermined event        includes detecting the predetermined event at the predetermined        position which is included in the image captured by the first        imaging apparatus.

11. The video surveillance method according to 8 or 9,

-   -   in which the step of detecting the predetermined event includes        detecting the predetermined event at another predetermined        position which is included in the image captured by the first        imaging apparatus and is different from the predetermined        position.

12. The video surveillance method according to 11, further including:

-   -   referring to a correspondence storage unit that stores a        plurality of pieces of correspondence relationship information        of a predetermined detection target position at which the        predetermined event is detected, and a predetermined        surveillance target position which corresponds to the        predetermined detection target position from among a plurality        of predetermined surveillance target positions; and    -   specifying the second imaging apparatus capable of capturing an        image of the predetermined surveillance target position        corresponding to the predetermined detection target position at        which the predetermined event has been detected.

13. The video surveillance method according to any one of Nos. 8 to 12,

-   -   in which the step of detecting the predetermined event includes        detecting a target object's passing across a video surveillance        line or a predetermined situation of the target object in a        surveillance region as the predetermined event.

14. The video surveillance method according to 13,

-   -   in which the step of detecting the predetermined event includes        detecting an occurrence of the target object's passing across        the video surveillance line or the predetermined situation of        the target object in the surveillance region for a plurality of        target objects during a predetermined time interval as the        predetermined event.

15. A program for causing at least one computer to execute the videosurveillance method according to any one of 8 to 14.

1. A video surveillance system comprising: at least one memory storinginstructions; and at least one processor configured to execute theinstructions to: detect, based on an image captured by a first imagingapparatus, an event indicating that a predetermined situation hasoccurred in a predetermined number of objects or more; and control asecond imaging apparatus to execute a cooperative action with the firstimaging apparatus when the predetermined event is detected.
 2. The imageprocessing system according to claim 1, wherein the at least oneprocessor is configured to execute the instructions to further detect,as the event, a number of persons passing across a video surveillanceline per unit time is greater than the predetermined number.
 3. Theimage processing system according to claim 1, wherein the at least oneprocessor is configured to execute the instructions to further controlthe second imaging apparatus to capture a surveillance target regioncorresponding to a detection target region in which the event isdetected.
 4. The image processing system according to claim 3, whereinthe at least one processor is configured to execute the instructions tofurther control the second imaging apparatus to change an imagecapturing direction of the second imaging apparatus so that the secondimaging apparatus can capture the surveillance target region.
 5. A videosurveillance method executed by a computer, the video surveillancemethod comprising: detecting, based on an image captured by a firstimaging apparatus, an event indicating that a predetermined situationhas occurred in a predetermined number of objects or more; andcontrolling a second imaging apparatus to execute a cooperative actionwith the first imaging apparatus when the predetermined event isdetected.
 6. The video surveillance method according to claim 5, furthercomprising detecting, as the event, a number of persons passing across avideo surveillance line per unit time is greater than the predeterminednumber.
 7. The video surveillance method according to claim 5, furthercomprising controlling the second imaging apparatus to capture asurveillance target region corresponding to a detection target region inwhich the event is detected.
 8. The video surveillance method accordingto claim 7, further comprising controlling the second imaging apparatusto change an image capturing direction of the second imaging apparatusso that the second imaging apparatus can capture the surveillance targetregion.
 9. A non-transitory computer-readable medium storing a programfor causing a computer to perform operations, the operations comprising:detecting, based on an image captured by a first imaging apparatus, anevent indicating that a predetermined situation has occurred in apredetermined number of objects or more; and controlling a secondimaging apparatus to execute a cooperative action with the first imagingapparatus when the predetermined event is detected.
 10. Thenon-transitory computer-readable medium according to claim 9, whereinthe operations further comprise detecting, as the event, a number ofpersons passing across a video surveillance line per unit time isgreater than the predetermined number.
 11. The non-transitorycomputer-readable medium according to claim 9, wherein the operationsfurther comprise controlling the second imaging apparatus to capture asurveillance target region corresponding to a detection target region inwhich the event is detected.
 12. The non-transitory computer-readablemedium according to claim 11, wherein the operations further comprisecontrolling the second imaging apparatus to change an image capturingdirection of the second imaging apparatus so that the second imagingapparatus can capture the surveillance target region.